Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import data from OpenSSF Malicious Packages #1412

Conversation

shravankshenoy
Copy link
Contributor

Fixes #1409

Changes Made

Created OpenSSF importer openssf.py and added files for testing the importer

Results

Ran the importer locally using command below with debug mode on
docker-compose exec vulnerablecode ./manage.py import vulnerabilities.importers.openssf.OpenSSFImporter

Importer ran successfully and imported 17182 advisories as can be seen in the image below

image

Other Consideration

  1. Initially I named the file ossf.py but it was tantalizingly close to oss_fuzz.py and could cause confusion, hence changed the name to openssf.py. Let me know if any better name is possible
  2. Out of the 4 ecosystems in the repository (crates.io, npm, pypi, rubygems), univers supports 3 (npm, pypi, rubygems), but I am not sure if it supports crates.io
  3. For the time being, I have added the 4 ecosystems in the list, but if required I can create a separate classmethod called supported_ecosystems and add all the ecosystems there as a list/dictionary, similar to how it is done in Vulntotal. Do let me know what is best approach
@classmethod
    def supported_ecosystem(cls):
        return {           
            "pypi": "PIP",
            "gem": "RUBYGEMS",
            "npm": "NPM",           
        }
  1. Have add a try-except block in for loop. Let me know if that has to be removed.

Copy link
Contributor

@TG1999 TG1999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shravankshenoy Thanks++, added some comments for your consideration

vulnerabilities/importers/openssf.py Outdated Show resolved Hide resolved
@@ -0,0 +1,9 @@
{
"aliases": ["MAL-2023-1077"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be used as an alias? What other options we have?

cc @pombredanne

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TG1999 yes, as an alias sounds fine

@mjherzog mjherzog changed the title Import data from OpenSSF Malacious Packages Import data from OpenSSF Malicious Packages Feb 5, 2024
@shravankshenoy
Copy link
Contributor Author

I have made the changes that had been requested. Let me know if anything else needs to be done.

@shravankshenoy
Copy link
Contributor Author

shravankshenoy commented Feb 12, 2024

While working on PRs #1405 and #1417 I realized that in this PR I have not followed the purl-spec properly. For example crates.io should be mapped to cargo and rubygems should be mapped to gems. I will work on making these changes.

@TG1999 TG1999 requested a review from ziadhany July 9, 2024 15:39
@TG1999
Copy link
Contributor

TG1999 commented Jul 22, 2024

@shravankshenoy Thanks! PR can not be merged in this state, I tried to run the importer and it's throwing errors. Please run the importer and improver once.

@TG1999 TG1999 closed this Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add malware from https://github.com/ossf/malicious-packages
3 participants